Electronic device and electronic device control meihod

ABSTRACT

An electronic device is disclosed. The disclosed electronic device comprises: a camera module; one or more processors electrically connected to the camera module; and a memory electrically connected to the processors, wherein, when the electronic device operates, the memory can store instructions for making the processors generate an image by using the camera module, calculate angles formed by a virtual horizontal line and a plurality of straight lines included in the image, and select two straight lines on the basis of the calculated angles. Additional various examples are possible.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a National Phase Entry of PCT InternationalApplication No. PCT/KR2018/002427, which was filed on Feb. 28, 2018, andclaims a priority to Korean Patent Application No. 10-2017-0027787,which was filed on Mar. 3, 2017, the contents of which are incorporatedherein by reference.

TECHNICAL FIELD

Various embodiments of the disclosure relate to an operating method ofdetecting a lane in which a transport apparatus is diving using anelectronic device and an apparatus performing the operation.

BACKGROUND ART

Recently, a diving assistance system is mounted on a transport apparatusin order to help a drivers decision to improve the ability to handle anaccident. Particularly, the number of transport apparatuses increases,which have a function for detecting the lane in which the transportapparatus is driving using a camera module mounted on the transportapparatus and controlling the operation of the transport apparatus whenthe transport apparatus deviates from the lane or notifying a driver ofthe deviation.

Furthermore, the number of applications, such as a driving assistancesystem, also increases in an electronic device on which a camera moduleis mounted, such as a smartphone. Accordingly, if a driving assistancesystem is not present in a transport apparatus, users may easily obtaininformation related to driving using smartphones.

DISCLOSURE OF INVENTION Technical Problem

If a function for detecting a lane is performed using an electronicdevice on which a camera module is mounted, such as a smartphone, a lanedetected by the smartphone may not be identical with the lane in which atransport apparatus is driving depending on the location where thesmartphone is mounted on the transport apparatus and the directiontoward the camera module included in the smartphone is directed.

The disclosure may provide a method capable of accurately detecting thelane in which a transport apparatus is diving although the locationwhere an electronic device including a camera module is positioned isout of the center of the transport apparatus and the direction towardwhich the camera module is directed is not identical with the directionin which the transport apparatus is diving, and an electronic devicesupporting the method.

Solution to Problem

According to an embodiment, an electronic device includes a cameramodule, at least one processor electrically connected to the cameramodule, and a memory electrically connected to the processor. When theelectronic device operates, the memory may store instructions enablingthe processor to generate an image using the camera module, to calculateangles formed by a plurality of straight lines and a virtual horizontalline included in the image, and to select two straight lines based onthe calculated angles.

According to an embodiment, a method of controlling an electronic devicemay include generating an image using a camera module, detecting aplurality of straight lines included in the generated image, andcalculating angles formed by the plurality of straight lines and avirtual horizontal line and selecting two straight lines based on thecalculated angles.

Advantageous Effects of Invention

According to an embodiment of the disclosure, a plurality of straightlines included in an image generated using a camera module is detected,and angles formed by the plurality of straight lines and a virtualhorizontal line are calculated. Two angles that belong to the calculatedangles and that are a preset value or less in the difference between theabsolute values of the two angles are calculated in order of greaterabsolute value. The virtual horizontal line and two straight linesforming the two angles are selected, and a face including the twostraight lines is selected as a lane. Accordingly, an electronic devicecan detect the lane in which a transport apparatus is driving althoughthe direction of the camera module is not identical with the directionin which the transport apparatus is driving.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1A is a diagram for illustrating the situation in which data forthe surrounding situation of a vehicle is collected using varioussensors included in a transport apparatus and is incorporated into avehicle operation.

FIG. 1B shows diagrams in which a transport apparatus according to anembodiment detects the lane in which the transport apparatus is drivingbased on an image received through a camera module, that is, one offunctions included in a driving assistance system.

FIG. 1C illustrates the case where the direction toward which the cameramodule of an electronic device according to an embodiment is notidentical with the direction in which a transport apparatus drives.

FIG. 1D is a diagram for illustrating the situation in which anelectronic device according to an embodiment detects the lane in which atransport apparatus on which the electronic device is put is driving byanalyzing an input image.

FIG. 2 is a block diagram of an electronic device according to variousembodiments.

FIG. 3 shows diagrams for illustrating the situation in which theelectronic device according to an embodiment detects a straight linecomponent in an input image.

FIG. 4 shows diagrams for illustrating the situation in which theelectronic device according to an embodiment detects the lane in which atransport apparatus is driving in an input image and changes an area forvehicle detection.

FIG. 5 is a flowchart illustrating a process for the electronic deviceaccording to an embodiment to detect a lane in an input image.

FIG. 6 is a diagram illustrating an electronic device within a networkenvironment according to various embodiments.

FIG. 7 is a block diagram of a program module according to variousembodiments.

MODE FOR THE INVENTION

Hereinafter, exemplary embodiments of the disclosure are described indetail with reference to the accompanying drawings. While the disclosuremay be embodied in many different forms, specific embodiments of thedisclosure are shown in drawings and are described herein in detail,with the understanding that the disclosure is to be considered as anexemplification of the principles of the invention and is not intendedto limit the invention to the specific embodiments illustrated. The samereference numbers are used throughout the drawings to refer to the sameor like parts. In the disclosure, an expression “or” includes anycombination or the entire combination of together listed words. Forexample, “A or B” may include A, B, or A and B. An expression of a firstand a second in the disclosure may represent various elements of thedisclosure, but do not limit corresponding elements. For example, theexpression does not limit order and/or importance of correspondingelements. The expression may be used for distinguishing one element fromanother element. For example, both a first user device and a second userdevice are user devices and represent different user devices. Forexample, a first constituent element may be referred to as a secondconstituent element without deviating from the scope of the disclosure,and similarly, a second constituent element may be referred to as afirst constituent element. When it is described that an element is“coupled” to another element, the element may be “directly coupled” tothe other element or “electrically coupled” to the other element througha third element. However, when it is described that an element is“directly coupled” to another element, no element may exist between theelement and the other element.

In this document, “configured (or set) to ˜” may be interchangeably usedwith “suitable for ˜” in hardware or software, “having the capacity to˜”, “adapted to ˜”, “made to ˜”, “capable of ˜” or “designed to ˜”according to circumstances. In some circumstances, an expression “deviceconfigured to ˜” may mean that the device “may perform ˜” along withother devices or parts. For example, a phrase “processor configured (orset) to perform A, B and C” may mean a dedicated processor (e.g., anembedded processor) for performing a corresponding operation or ageneric-purpose processor (e.g., a CPU or an application processor)capable of performing corresponding operations by executing one or moresoftware programs stored in a memory device.

An electronic device according to various embodiments of this documentmay include at least one of a smart phone, a tablet PC, a mobile phone,a video phone, an e-book reader, a desktop PC, a laptop PC, a netbookcomputer, a workstation, a server, a PDA, a portable multimedia player(PMP), an MP3 player, a medical device, a camera, or a wearable device.The wearable device may include at least one of an accessory type (e.g.,a watch, a ring, a bracelet, an ankle bracelet, a necklace, glasses, acontact lens or a head-mounted-device (HMD)), a fabric orclothing-integrated type (e.g., electronic clothing), a body-attachedtype (e.g., a skin pad or tattoo) or a bio-transplant type. In someembodiments, the electronic device may include at least one oftelevision, a digital video disk (DVD) player, an audio, a set-top box,a home automation control panel, a security control panel, a media box(e.g., Samsung HomeSync™, Apple TV™ or Google TV™), a game console(e.g., Xbox™, PlayStation™), an electronic dictionary, an electronickey, a camcorder or an electronic picture frame, for example.

In another embodiment, the electronic device may include at least one ofa navigation device, a satellite navigation system (global navigationsatellite system (GNSS)), an event data recorder (EDR), a flight datarecorder (FDR), a vehicle infotainment device, electronic equipment forship (e.g., a navigator for ship and a gyro compass), avionics, securitydevices, a head unit for vehicle, industry or home robots, a drone orInternet of things. In various embodiments, the electronic device may beflexible or may be a combination of two or more of the various devices.The electronic device according to an embodiment of this document is notlimited to the above-described devices. In this document, a term “user”may refer to a person who uses the electronic device or a device (e.g.,an artificial intelligence electronic device) that uses the electronicdevice.

(a) of FIG. 1A and (b) of FIG. 1A are diagrams for illustrating thesituation in which is data for the surrounding situation of a vehicle iscollected using various sensors included in the vehicle and incorporatedinto a vehicle operation.

Referring to FIG. 1A, a transport apparatus 10 may provide safety andconvenience to a driver who drives the transport apparatus 10 or apassenger using various sensors. The transport apparatus 10 may controlthe driving of the transport apparatus 10 based on information collectedfrom various sensors using a processor included in the transportapparatus 10 or electrically connected to the transport apparatus 10, ormay display driving guidance information to a user using at least onedisplay or notification sensor included in the transport apparatus 10.Such a function may be called a driving assistance system (e.g., anadvanced driver assistance system (ADAS)).

In order for the diving assistance system to perform a proper operationbased on the condition of the transport apparatus 10, various sensorscapable of recognizing the inside or outside state of the transportapparatus 10 may be necessary. A sensor for the driving assistancesystem may be various depending on its role in the transport apparatus10, but may approximately include radio detection and ranging (RADAR),light amplification by the stimulated of radiation (LASER), ultrasonicwaves, a camera module, and an acceleration/angular speed/steering anglesensor.

A radar sensor 20 may obtain driving information, such as the distance,height, direction, and speed, by analyzing a signal reflected after anelectromagnetic wave is radiated toward an object, for example. Thetransport apparatus 10 may perform functions, such as automatic drivingcontrol, front/rear collision warning, blind spot detection, andautomatic emergency braking, using the radar sensor 20.

The radar sensor 20 may maintain its detection ability regardless of adriving environment (e.g., rain driving, night driving), and mayrecognize an object at the distance of about 200 m, for example.

The laser sensor may emit a photon, that is, a particle of light, in afrequency of a narrow band. The laser sensor may obtain surroundinginformation by analyzing light reflected from an object after it emitsultraviolet rays or infrared rays (infrared), for example.

The laser sensor may maintain its detection ability even at nightdriving, but has a disadvantage of a short recognition distance.

An ultrasonic sensor 30 may calculate the distance from an object bymeasuring the time that a sound wave of a high frequency (e.g., 20 kHz),for example, is taken to be returned after the sound wave is reflectedby an object. The sound wave is not faster in speed than light, and thusmay be chiefly used when the transport apparatus 10 is parked or whenthe distance from a surrounding object is measured while the transportapparatus moves at a low speed.

A camera module 40 has different resolution, precision, etc. of acaptured image depending on a photographing environment, but has anadvantage in that it can recognize a shape of an object. Accordingly,the camera module 40 may be used for functions, such as lane detection,rear-end collision prevention, and full beam control, in the transportapparatus 10.

The above-described sensors may be basically sensors for recognizing asurrounding environment outside the transport apparatus 10. In order toincrease the accuracy and utilization of a driving assistance system, asensor for detecting a motion state of the transport apparatus 10 itselfis also important. The transport apparatus 10 may be capable ofmeasures, such as braking and steering, based on the results ofdetection of a sensor only when a motion state of the transportapparatus 10 itself is detected.

A sensor 50 for detecting a motion state of the transport apparatus mayinclude an acceleration sensor, an angular velocity sensor, and asteering angle sensor, for example.

The acceleration sensor may instantaneously detect a dynamic force, suchas an acceleration force, vibration or an impact that acts on an object.The acceleration sensor may detect the shaking, direction, etc. of anobject. Accordingly, if the acceleration sensor is used, the transportapparatus 10 may detect a movement between specific points. If theangular velocity sensor is used, the rotation of the transport apparatus10 may also be detected. The steering angle sensor may obtain the angleof a steering wheel included in the transport apparatus 10. Thetransport apparatus 10 may use the sensors related to the motion statefor driving posture control and an airbag operation.

(b) of FIG. 1A shows a form seen in a divers view within the transportapparatus 10 when the driver drives the transport apparatus 10. Thedriver may view a situation outside the transport apparatus 10 throughthe windshield glass of the transport apparatus 10, for example, and mayalso view a steering wheel for driving, a cluster screen, and othermanipulation devices. The transport apparatus 10 may display drivingguidance information, generated using the above-described varioussensors, on a display 12. Furthermore, if the transport apparatus 10includes a head up display (HUD), the transport apparatus may displaydriving guidance information on the HUD.

The transport apparatus 10 may check the lane in which the transportapparatus is driving and the presence or absence of another vehiclepositioned in the driving lane using an image received through thecamera module 40, and may display the lane or another vehicle on thedisplay 10, if necessary. Accordingly, a driver can drive the transportapparatus 10, while watching driving guidance information generated bythe transport apparatus.

Furthermore, the transport apparatus 10 may control the driving of thetransport apparatus 10 using driving-related information generated usingvarious sensors. For example, the transport apparatus 10 may reduce thespeed of the transport apparatus 10 based on the distance from a vehicleahead. If the transport apparatus 10 deviates from a driving lanewithout a separate signal, it may generate and output an alarm.

FIG. 1B is a diagram in which a transport apparatus according to anembodiment detects the lane in which the transport apparatus is drivingbased on an image received through a camera module (e.g., the cameramodule 40 of FIG. 1A), that is, one of functions included in a drivingassistance system.

Referring to (a) of FIG. 1B, the transport apparatus 10 may detect thelane in which the transport apparatus 10 is driving based on an imagereceived through a camera module (e.g., the camera module 40 of FIG. 1A)included in the transport apparatus 10.

For example, the transport apparatus 10 may previously set a referencevanishing point 61, a straight line 62 corresponding to a left lane, astraight line 63 corresponding to a right lane, a vehicle hood 64, avalid lane width range 66, and a vehicle and lane detection area 65.

The transport apparatus 10 according to an embodiment may perform lanedetection on an image, received through a camera module (e.g., thecamera module 40 of FIG. 1A), in the preset vehicle and lane detectionarea 65. Specifically, the transport apparatus 10 may receive a presetframe (e.g., 300 frames), and may detect an edge area in the receivedimage. Particularly, the transport apparatus 10 may detect edges inareas adjacent the preset reference vanishing point 61, the straightline 62 corresponding to the left lane, the straight line 63corresponding to the right lane, and the vehicle hood 64, and maycorrect the locations of the reference vanishing point 61, the straightline 62 corresponding to the left lane, the straight line 63corresponding to the right lane, and the vehicle hood 64.

Referring to (b) of FIG. 1B, the transport apparatus 10 may change alane and vehicle detection area 75, a valid lane width range 76, etc.based on the locations of a changed reference vanishing point 71,straight line 72 corresponding to the left lane, a straight line 73corresponding to the right lane, and a vehicle hood 74.

As described above, the transport apparatus 10 may detect a driving laneusing a camera module (e.g., the camera module 40 of FIG. 1A) mounted onthe transport apparatus 10, and may use the detected lane for a drivingassistance system. According to an embodiment, the transport apparatus10 may detect a driving lane using a separate electronic device.

Referring to (a) of FIG. 1C, a user may put an electronic device 100,such as a smartphone, on the transport apparatus 10, may detect a lanebased on an image generated through a camera module (not shown) includedin the electronic device 100, and may use the detected lane for a divingassistance system. For example, the electronic device 100 may detect alane based on the execution of an application for performing a drivingassistance system function. In this case, the electronic device 100 andthe transport apparatus 10 may communicate with each other using a wiredor wireless communication method.

Wireless communication may include cellular communication using at leastone of LTE, LTE advance (LTE-A), code division multiple access (CDMA),wideband CDMA (WCDMA), a universal mobile telecommunications system(UMTS), a wireless broadband (WiBro), or global system for mobilecommunications (GSM), for example. According to an embodiment, wirelesscommunication may include at least one of wireless fidelity (WiFi),Bluetooth, Bluetooth low energy (BLE), Zigbee, near field communication(NFC), magnetic secure transmission, a radio frequency (RF), or a bodyarea network (BAN), for example.

According to an embodiment, wireless communication may include a GNSS.The GNSS may be a global positioning system (GPS), a global navigationsatellite system (Glonass), a Beidou navigation satellite system(hereinafter referred to as “Beidou”), Galileo, or the European globalsatellite-based navigation system, for example. Hereinafter, in thisdocument, the “GPS” and the “GNSS” may be interchangeably used.

Wired communication may include at least one of a universal serial bus(USB), a high definition multimedia interface (HDMI), recommendedstandard 232 (RS-232), power line communication, or plain old telephoneservice (POTS), for example. A network may include at least one oftelecommunication networks, for example, a computer network (e.g., LANor WAN), Internet, or a telephone network.

Referring to (a) of FIG. 1C, if a user puts the electronic device 100 onthe transport apparatus 10, the electronic device 100 may not bepositioned at the central part of the transport apparatus 10.Furthermore, the direction A in which the transport apparatus 10 isdiving and the direction B toward which a camera module included in theelectronic device 100 is directed may not be the same. In this case, alane detected by the electronic device 100 may not be the lane in whichthe transport apparatus 10 is driving.

(b) of FIG. 1C shows an image which is obtained through a camera moduleincluded in the electronic device 100 and displayed on the display ofthe electronic device 100 or which is displayed on a display (not shown)included in the transport apparatus 10 through wired or wirelesscommunication. The electronic device 100 may detect the lane in whichthe transport apparatus 10 is driving using a method identical with orsimilar to the method described in FIG. 1B. For example, the electronicdevice 100 may detect a first lane 78 and a second lane 79 in the inputimage.

According to an embodiment, the electronic device 100 may recognize thesecond lane 79 that belongs to the detected lane and that is positionedat the center of the image as the lane in which the transport apparatus10 is driving. Accordingly, the electronic device 100 needs to change acriterion for selecting a lane that belongs to detected lanes and inwhich the transport apparatus 10 on which the electronic device 100 isput is driving.

FIG. 1D show diagrams for illustrating the situation in which theelectronic device 100 according to an embodiment detects the lane inwhich the transport apparatus 10 on which the electronic device 100 isput is driving by analyzing an input image.

Referring to (a) of FIG. 1D, the electronic device 100 may convert aninput image into an image that highlights an edge component. A detailedprocess of converting an input image into an image that highlights anedge is described later with reference to FIG. 3.

The electronic device 100 according to an embodiment may calculateangles formed by straight lines 81, 82, and 83, detected in a presetvehicle and lane detection area (e.g., the vehicle and lane detectionarea 65 of FIG. 1B), and a virtual horizontal line 85. For example, theelectronic device 100 may calculate a first angle 91 formed by the firststraight line 81 and the virtual horizontal line 85. Furthermore, theelectronic device 100 may calculate a second angle 92 formed by thesecond straight line 82 and the virtual horizontal line 85. Furthermore,the electronic device 100 may calculate may calculate a third angle 93formed by the third straight line 83 and the virtual horizontal line 85.The electronic device 100 may select two straight lines based on thecalculated angles.

For example, the electronic device 100 may extract two angles, having apreset value or less in the difference between their absolute values,from the calculated angles 91, 92, and 93 in order of greater absolutevalue, and may select the virtual horizontal line and two straight linesforming the extracted two angles.

Specifically, the electronic device 100 may extract the first angle 91and the second angle 92 that belong to the first angle 91, the secondangle 92, and the third angle 93 and that have a preset value or less inthe difference between their absolute values in order of greaterabsolute value. Furthermore, the electronic device 100 may select thevirtual horizontal line 85, and the first straight line 81 and thesecond straight line 82 that form the first angle 91 and the secondangle 92. If a difference between absolute values is a preset value orless, this may mean that a difference between the absolute values isabout 5 degrees to 10 degrees, for example, but the disclosure is notlimited thereto.

Referring to (b) of FIG. 1D, the electronic device 100 may select thelane 95, including the selected two straight lines, as the lane in whichthe transport apparatus 10 on which the electronic device 100 is put isdiving.

As described above, the electronic device 100 according to an embodimentmay establish communication with the transport apparatus 10 using wiredor wireless communication, and may provide some of the functions of adriving assistance system. Particularly, the electronic device 100according to an embodiment may detect the lane in which the transportapparatus 10 is driving although the direction of a camera moduleincluded in the electronic device 100 is not identical with thedirection in which the transport apparatus 10 is driving.

FIG. 2 is a block diagram illustrating an example electronic device 201in accordance with an example embodiment of the disclosure. Theelectronic device 201 may include at least one application processor(AP) (e.g., including processing circuitry) 210, a communication module(e.g., including communication circuitry) 220, a subscriberidentification module (SIM) card 224, a memory 230, a sensor module 240,an input device (e.g., including input circuitry) 250, a display 260, aninterface (e.g., including interface circuitry) 270, an audio module280, a camera module 291, a power management module 295, a battery 296,an indicator 297, and a motor 298. The application processor 210 maycontrol a plurality of hardware or software components connected theretoby driving an operating system or applications, for example, and mayperform various types of data processing and operation. The applicationprocessor 210 may be formed of system-on-chip (SoC), for example.According to an embodiment, the application processor 210 may furtherinclude a graphic processing unit (GPU) and/or an image processor. Theapplication processor 210 may include at least one (e.g., cellularmodule 221) of the elements shown in FIG. 2. The processor 210 may loadinstructions or data, received from at least one of other elements(e.g., nonvolatile memory), into a volatile memory, may process theinstructions or data, and may store resulting data in a nonvolatilememory.

The communication module 220 may include a cellular module 221, a WiFimodule 223, a Bluetooth module 225, a GNSS module 227, an NFC module228, and an RF module 229. The cellular module 221 may provide a voicecall, a video call, text services or Internet service. According to anembodiment, the cellular module 221 may perform the identification andauthentication of the electronic device in a communication network usinga subscriber identity module (e.g., SIM card 224). According to anembodiment, the cellular module 221 may perform at least some offunctions that may be provided by the AP 210. According to anembodiment, the cellular module 221 may include a communicationprocessor (CP). According to an embodiment, at least some (e.g., two ormore) of the cellular module 221, the WiFi module 223, the Bluetoothmodule 225, the GNSS module 227 or the NFC module 228 may be included ina single integrated circuit (IC) chip or a single IC package. The RFmodule 229 may transmit and receive data communication signals (e.g., RFsignals), for example. The RF module 229 may include a transceiver, apower amp module (PAM), a frequency filter, a low noise amplifier (LNA)or an antenna. According to another embodiment, at least one of thecellular module 221, the WiFi module 223, the Bluetooth module 225, theGNSS module 227 or the NFC module 228 may transmit and receive RFsignals through a separate RF module. The subscriber identity module 224may include a card including a subscriber identity module or an embeddedSIM, and may include unique identity information (e.g., integratedcircuit card identifier (ICCID) or subscriber information (e.g.,international mobile subscriber identity (IMSI).

The memory 230 may include an internal memory 232 or an external memory234. The internal memory 232 may include least one of a volatile memory(e.g., DRAM, SRAM or SDRAM), a nonvolatile memory (e.g., one timeprogrammable ROM (OTPROM), PROM, EPROM, EEPROM, mask ROM, flash ROM,NAND flash memory, a hard drive, or a solid state drive (SSD), forexample. The external memory 234 may include a flash drive, for example,a compact flash (CF), secure digital (SD), micro-SD, mini-SD, extremedigital (xD), a multi-media card (MMC) or a memory stick. The externalmemory 234 may be connected to the electronic device 100 functionally orphysically through various interfaces.

The sensor module 240 may measure physical quantity or detect theoperating status of the electronic device 100, and may convert measuredor sensed information into electric signals. The sensor module 240 mayinclude at least one of a gesture sensor 240A, a gyro sensor 240B, anatmospheric sensor 240C, a magnetic sensor 240D, an acceleration sensor240E, a grip sensor 240F, a proximity sensor 240G, a color sensor 240H(e.g., red, green, blue (RGB) sensor), a biometric sensor 240I, atemperature/humidity sensor 240J, an illumination sensor 240K, or aultraviolet (UV) sensor 240M, for example. Additionally oralternatively, the sensor module 240 may include an e-nose sensor, anelectromyography (EMG) sensor, an electroencephalogram (EEG) sensor, anelectrocardiogram (ECG) sensor, an infrared (IR) sensor, an iris scansensor and/or a finger scan sensor, for example. Furthermore, the sensormodule 240 may further include a control circuit for controlling one ormore sensors included therein. In an embodiment, the electronic device100 may further include a processor configured to control the sensormodule 240 as part of the processor 210 or separately, and may controlthe sensor module 240 while the processor 210 is in a sleep state.

The input device 250 may include a touch panel 252, a (digital) pensensor 254, a key 256 or an ultrasonic input unit 258, for example. Thetouch panel 252 may use at least one of a capacitive type, resistivetype, infrared type, or ultrasonic type, for example. Furthermore, thetouch panel 252 may further include a control circuit. The touch panel252 may further include a tactile layer, and may provide a tactileresponse to a user. The (digital) pen sensor 254 may be part of thetouch panel or may include a separate sheet for recognition. The key 256may include a physical button, an optical key, or a keypad, for example.The ultrasonic input unit 258 may identify data corresponding toultrasonic waves by detecting ultrasonic waves, generated from an inputtool, through a microphone 288.

The display 260 may include a panel 262, a hologram 264, a projector 266and/or a control circuit for controlling them. The panel 262 may beimplemented in such a way as to be flexible, transparent or wearable.The panel 262 may be configured with a touch panel 252 and one or moremodules. According to one embodiment, the panel 262 may include apressure sensor (or force sensor) capable of measuring the intensity ofpressure on a users touch. The pressure sensor may be integrated withthe touch panel 252 or may be implemented as a sensor separated from thetouch panel 252. The hologram 264 may show a stereoscopic image in theair using interference of light. The projector 266 may display an imageby projecting light onto a screen. The screen may be positioned insideor outside the electronic device 100. The interface 270 may include anHDMI 272, a USB 274, an optical interface 276 or a D-subminiature(D-sub) 278. Additionally or alternatively, the interface 270 mayinclude, for example, a mobile high-definition link (MHL) interface, asecure digital (SD) card/multi-media card (MMC) interface, or aninfrared data association (IrDA) interface.

The audio module 280 may convert sound and electric signals and viceversa. The audio module 280 may process sound information input oroutput through a speaker 282, a receiver 284, an earphone 286, or amicrophone 288. The camera module 291 is a device capable of obtainingstill images and moving images. According to an embodiment, the cameramodule may include at least one image sensor (e.g., a front sensor or arear sensor), a lens, an image signal processor (ISP), or a flash (e.g.,LED or xenon lamp). The power management module 295 may manage power ofthe electronic device 100. According to an embodiment, the powermanagement module 295 may include a power management integrated circuit(PMIC), a charger IC, or a battery or fuel gauge, for example. The PMICmay have a wired and/or wireless charging method. The wireless chargingmethod may include a magnetic resonance type, a magnetic induction type,or an electromagnetic type, for example, and may include additionalcircuits for wireless charging, for example, a coil loop, a resonancecircuit, or a rectifier. The battery gauge may measure the residualamount of the battery 296 and a voltage, current or temperature duringcharging. The battery 296 may include a rechargeable battery and/or asolar battery, for example.

The indicator 297 may display a given status (e.g., a booting status, amessage status, or a charging status) of the electronic device 100 or apart thereof (e.g., the AP 210). The motor 298 may convert an electricsignal into mechanical vibration, and may generate vibration or a hapticeffect. The electronic device 100 may include a mobile TV support device(e.g., GPU) capable of processing media data according to the standardsof digital multimedia broadcasting (DMB), digital video broadcasting(DVB) or mediaFlo™. Each of the elements described in this document maybe configured with one or more components, and the name of acorresponding component may be different depending on the type ofelectronic device. In various embodiments, an electronic device (e.g.,the electronic device 100) may omit some of the components or mayfurther include an additional component or some of the components may becombined into a single entity, which may perform the same function asthose of the component prior to the combination.

FIG. 3 shows diagrams for illustrating the situation in which theelectronic device according to an embodiment changes an input image intoan image that highlights an edge in order to detect a straight linecomponent in the input image.

The electronic device 100 according to an embodiment may analyze aninput image through a camera module (not shown) included in theelectronic device 100 in order to detect the lane in which a transportapparatus (e.g., the transport apparatus 10 of FIG. 1A) on which theelectronic device 100 is put is driving.

Referring to (a) of FIG. 3 and (b) of FIG. 3, the electronic device 100may perform a gray scale on an input image through the camera module.The gray scale may mean that a generated color image is converted intoan image having white to black of 10 steps, for example. That is, theelectronic device 100 may convert a color image into an image havingonly a brightness value without color information.

The electronic device 100 according to an embodiment may detect an edgeusing Sobel operation having high efficiency in edge detection of adiagonal direction in order to extract a lane from an image, but thedisclosure is not limited thereto. A method of calculating a slope in animage and a method using a mask, that is, detection methods based on thecalculation of a partial differentiation operator, may be used as amethod for the electronic device 100 to detect an edge using Sobeloperation.

Referring to (b) of FIG. 3 and (c) of FIG. 3, the electronic device 100may perform a binarization task on an edge component, extracted by Sobeloperation, in order to remove noise included in the image. Thebinarization task is operation of changing a pixel value of an imageinto 0 or 255, for example, and may be a task for dividing theattributes of a pixel into two groups of a background and an object. Theelectronic device 100 may obtain information of a given object (e.g.,the location, size, and shape of an object) within the image using thebinarization task.

According to an embodiment, the electronic device 100 may perform Houghtransformation on the image from which noise has been removed in orderto find out a straight line for lane detection. The boundary line of anobject can be found out through the Hough transformation although thereis no prior knowledge on the object included in an image. Accordingly,as in (C) of FIG. 3, the electronic device 100 may add up pixel valuesof a plurality of straight line candidates included in an image usingHough transformation, may detect a maximum value of the pixel values,and may detect one straight line.

An algorithm used in the above-described image transform is merely anembodiment, and may be substituted with another algorithm depending onthe characteristics of an image and the characteristics of a processor.

According to an embodiment, the electronic device 100 may perform theprocess on multiple image frames configuring an image that is receivedthrough a camera module and generated. For example, if the camera moduleof the electronic device 100 generates an image of 30 frames per second,the image processing process may be performed on each of the generated30 frames, but the disclosure is not limited thereto.

The electronic device 100 may obtain an image, such as (a) of FIG. 1Dand may obtain angles formed by a detected straight line component and avirtual horizontal line (e.g., the horizontal line 85 of FIG. 1D)through the above step.

FIG. 4 shows diagrams for illustrating the situation in which anelectronic device according to an embodiment detects the lane in which atransport apparatus is driving in an input image and then changes anarea for vehicle detection.

Referring to (a) of FIG. 4, the electronic device 100 may set an objectdetection area 410 for detecting a vehicle ahead at the central part ofan input image. The area for object detection may be displayed at thecentral part of the input image, for example. The electronic device 100may detect whether a vehicle is present by analyzing an image of theobject detection area 410 for detecting a vehicle ahead. The objectdetection area 410 for detecting a vehicle ahead may have a squareshape, for example. According to an embodiment, the electronic device100 may detect the presence or absence of a vehicle and transmit aresult of the detection to a transport apparatus (e.g., the transportapparatus 10 of FIG. 1A). The transport apparatus (e.g., the transportapparatus 10 of FIG. 1A) may control the driving of the transportapparatus (e.g., the transport apparatus 10 of FIG. 1A) or generatenotification for a user by determining the distance from a vehicle aheaddetected using the radar sensor described in FIG. 1A.

The electronic device 100 may set the object detection area 410 fordetecting a vehicle ahead at the central part, assuming that thedirection in which a transport apparatus (e.g., the transport apparatus10 of FIG. 1A) proceeds and the direction toward which a camera moduleincluded in the electronic device 100 is directed are the same. However,if the direction toward which a camera module included in the electronicdevice 100 is directed and the direction in which a transport apparatus(e.g., the transport apparatus 10 of FIG. 1A) put on the electronicdevice 100 drives are the same as in (a) of FIG. 10, the objectdetection area 410 for detecting a vehicle ahead and the lane in whichthe transport apparatus (e.g., the transport apparatus 10 of FIG. 1A) isdriving may not be the same.

Referring to (b) of FIG. 4, the electronic device 100 may detect thelane in which a transport apparatus (e.g., the transport apparatus 10 ofFIG. 1A) on which the electronic device 100 is put is driving using themethod described in FIG. 1D.

According to an embodiment, the electronic device 100 may set a centerpoint below a first straight line 421 and second straight line 422determined to be a lane in which a transport apparatus (e.g., thetransport apparatus 10 of FIG. 1A) is driving as a drivers lanereference point 430.

According to an embodiment, the electronic device 100 may select a area423, including the first straight line 421 and the second straight line422, as the lane in which a transport apparatus (e.g., the transportapparatus 10 of FIG. 1A) is driving. The electronic device 100 may setthe center point 430 of the lane in which the transport apparatus 10 isdriving. The electronic device 100 may set a vertical line 431 passingthrough the center point 430, and may set the object detection area 410so that it is bisected by the vertical line 431 and corners 411 and 412at the bottom of the object detection area 410 neighbor the firststraight line 421 and the second straight line 422, respectively.However, the disclosure is not limited thereto. The electronic device100 may generate the object detection area 410 having various shapes andsizes.

According to another embodiment, if the electronic device 100 has notset the object detection area 410, it may set a location of the objectdetection area 410 that has not yet been determined using theabove-described method.

Accordingly, the electronic device 100 can detect a transport apparatusin a driving lane with high accuracy although the direction toward whicha camera module is directed is not the same as the direction in which atransport apparatus (e.g., the transport apparatus 10 of FIG. 1A) isrunning. Furthermore, the electronic device 100 may reduce a detectiontime by detecting an object only in some of an image.

FIG. 5 is a flowchart illustrating a process for an electronic deviceaccording to an embodiment to detect a lane in an input image.

Referring to operation 510, the electronic device 100 may generate aninput image through a camera module. According to an embodiment, theelectronic device 100 may include the camera module, and may establish acommunication relation with a separate camera module and receive datarelated to an image from the camera module.

Referring to operation 520, the electronic device 100 may detect astraight line component included in the generated image. The electronicdevice 100 may detect the straight line component using various imageprocessing methods. For example, the electronic device may detect thestraight line component by performing image processing, such as a grayscale task, Sobel operation, a binarization task, or a Houghtransformation task, on the input image. However, the task performed bythe electronic device 100 in order to detect the straight line componentis not limited thereto, and may include various methods depending on thecharacteristics of an image and the characteristics of an electronicdevice.

Furthermore, the electronic device 100 may detect a plurality ofstraight line components in the entire generated image or may detect astraight line component only in a preset area.

Referring to operation 530, the electronic device 100 may calculateangles formed by a plurality of detected straight line components and avirtual horizontal line, and may select two straight lines based on thecalculated angles. For example, the electronic device 100 may select twoangles, having a preset value or less in a difference between theabsolute values of the calculated angles, in order of greater absolutevalue, and may select the above-described virtual horizontal line andthe two straight lines forming the two angles. The electronic device 100may select a lane, including the selected two straight lines, as thelane in which a transport apparatus on which the electronic device 100is put is driving.

Referring to FIG. 6, there is described an electronic device 2101 withina network environment 2100 in various embodiments. The electronic device2101 may include the electronic device 100 of FIG. 2. The electronicdevice 2101 may include a bus 2110, a processor 2120, a memory 2130, aninput and output interface 2150, a display 2160, and a communicationinterface 2170. In an embodiment, the electronic device 2101 may omit atleast one of the elements or may additionally include another element.The bus 2110 may include a circuit for connecting the elements 2110-2170and delivering communication (e.g., a control message or data) betweenthe elements. The processor 2120 may include one or more of a centralprocessing unit, an application processor, or a communication processor(CP). The processor 2120 may execute operation or data processingregarding control and/or communication of at least another element ofthe electronic device 2101, for example.

The memory 2130 may include a volatile memory and/or a nonvolatilememory. The memory 2130 may store instructions or data related to atleast one element of the electronic device 2101, for example. Accordingto one embodiment, the memory 2130 may store software and/or a program2140. The program 2140 may include, for example, a kernel 2141,middleware 2143, an application programming interface (API) 2145 and/oran application program (or “application”) 2147. At least some of thekernel 2141, the middleware 2143 or the API 2145 may be called anoperating system (OS). The kernel 2141 may control or manage systemresources (e.g., the bus 2110, the processor 2120 and the memory 2130)used to execute operations or functions implemented in other programs(e.g., the middleware 2143, the API 2145 or the application program2147), for example. Furthermore, the kernel 2141 may provide aninterface capable of controlling or managing system resources byaccessing individual elements of the electronic device 2101 in themiddleware 2143, the API 2145 or the application program 2147.

The middleware 2143 may perform a mediation role so that the API 2145 orthe application program 2147, for example, can exchange data throughcommunication with the kernel 2141. Furthermore, the middleware 2143 mayprocess one or more task requests received from the application program2147 based on priority. For example, the middleware 2143 may assignpriority on which the system resources (e.g., the bus 2110, theprocessor 2120 and the memory 2130) of the electronic device 2101 can beused to at least one of the application programs 2147, and may processthe one or more task requests. The API 2145 is an interface throughwhich the application 2147 controls a function provided by the kernel2141 or the middleware 2143, and may include at least one interface orfunction (e.g., an instruction) for file control, window control, imageprocessing or text control. The input/output interface 2150 may deliveran instruction or data received from a user or another external deviceto other element(s) of the electronic device 2101 or may output aninstruction or data, received from other element(s) of the electronicdevice 2101 to a user or another external device, for example.

The display 2160 may include, for example, a liquid crystal display(LCD), a light-emitting diode (LED) display, an organic light-emittingdiode (OLED) display, a microelectromechanical systems (MEMS) display oran electronic paper display. The display 2160 may display, for example,various types of content (e.g., text, images, video, icons and symbols)to a user. The display 2160 may include a touch screen, and may receive,for example, a touch, a gesture, proximity or hovering input using anelectronic pen or part of the body of a user. The communicationinterface 2170 may establish communication between the electronic device2101 and an external device (e.g., the first external communicationinterface 2102, the second external electronic device 2104, or theserver 2106), for example. For example, the communication interface 2170may be connected to the network 2162 through wireless communication orwired communication, and may communicate with an external device (e.g.,the second external electronic device 2104 or the server 2106).

Each of the first and second external electronic devices 2102 and 2104may be a device having the same type as the electronic device 2101 orhaving a type different from the type of the electronic device 2101. Inaccordance with various embodiments, some or all of operations executedin the electronic device 2101 may be executed in another one or aplurality of electronic devices (e.g., the electronic devices 2102 and2104 and the server 2106). According to one embodiment, if theelectronic device 2101 has to perform a function or serviceautomatically or in response to a request, the electronic device 2101may execute the function or service or additionally request at leastsome associated functions from another device (e.g., the electronicdevice 2102, 2104 or the server 2106). Another electronic device (e.g.,the electronic device 2102, 2104 or the server 2106) may execute arequested function or additional function and transfer the resultsthereof to the electronic device 2101. The electronic device 2101 mayprovide the received results as a requested function or service withoutany change or may provide the requested function or service byadditionally processing the received results. To this end, for example,clouding computing, distributed computing or client-server computingtechnology may be used.

FIG. 7 is a block diagram of a program module according to variousembodiments of the present disclosure. In accordance with oneembodiment, the program module 2310 (e.g., program 2140) may include anoperating system (OS) that controls resources related to an electronicdevice (e.g., the electronic device 2101) and/or various applications(e.g., the application program 2147) driven on the operating system. Theoperating system may include, for example, Android™, Windows™, Symbian™,Tizen™ or Bada™. Referring to FIG. 7, the program module 2310 mayinclude a kernel 2320 (e.g., the kernel 2141), middleware 2330 (e.g.,the middleware 2143), an API 2360 (e.g., the API 2145) and/or anapplication 2370 (e.g., the application program 2147). At least some ofthe program module 2310 may be pre-loaded onto the electronic device ormay be downloaded from an external electronic device (e.g., theelectronic device 2102, 2104 or the server 2106.

The kernel 2320 may include, for example, a system resource manager 2321and/or a device driver 2323. The system resource manager 2321 mayperform control, allocation or recovery of system resources. Inaccordance with one embodiment, the system resource manager 2321 mayinclude a process management unit, a memory management unit or a filesystem management unit. The device driver 2323 may include, for example,a display a driver, a camera driver, a Bluetooth driver, sharing memorydriver, a USB driver, a keypad driver, a Wi-Fi driver, an audio driveror an inter-process communication (IPC) driver. The middleware 2330 mayprovide, for example, a function required by the application 2370 incommon or may provide various functions to the application 2370 so thatthe application 2370 can use limited system resources within theelectronic device through the API 2360. In accordance with oneembodiment, the middleware 2330 may include at least one of a runtimelibrary 2335, an application manager 2341, a window manager 2342, amultimedia manager 2343, a resource manager 2344, a power manager 2345,a database manager 2346, a package manager 2347, a connectivity manager2348, a notification manager 2349, a location manager 2350, a graphicmanager 2351, or a security manager 2352.

The runtime library 2335 may include a library module used by a compilerin order to add a new function through a programming language while theapplication 2370 is executed, for example. The runtime library 2335 mayperform a function for input/output management, memory management orarithmetic function. The application manager 2341 may manage the lifecycle of the application 2370, for example. The window manager 2342 maymanage GUI resources used by a screen. The multimedia manager 2343 mayidentify a format necessary for the playback of various media files andperform the encoding or decoding of a media file using a codec suitablefor a corresponding format. The resource manager 2344 may manage thesource code or the space of memory. The power manager 2345 may managethe capacity of the battery or power, for example, and may provide powerinformation necessary for the operation of the electronic device. Inaccordance with one embodiment, the power manager 2345 may operate inconjunction with a basic input/output system (BIOS). The databasemanager 2346 may generate, search or change a database to be used in theapplication 2370. The package manager 2347 may manage the installationor update of an application distributed in the form of a package file.

The connectivity manager 2348 may manage a wireless connection, forexample. The notification manager 2349 may provide a user with an event,such as a received message, an appointment or proximity notification.The location manager 2350 may manage location information of theelectronic device, for example. The graphic manager 2351 may manage agraphic effect to be provided to a user or a user interface related tothe graphic effect. The security manager 2352 may provide systemsecurity or user authentication. In accordance with one embodiment, themiddleware 2330 may include a telephony manager for managing the voiceor video communication function of the electronic device or a middlewaremodule capable of forming a combination of the functions of theelements. In accordance with one embodiment, the middleware 2330 mayprovide a module specified for each type of operating system. Themiddleware 2330 may dynamically delete some of the existing elements oradd new elements. The API 2360 is a set of API programming functions,for example, and may be provided as a different element depending on anoperating system. For example, the API may provide one API set for eachplatform in the case of Android or iOS, and may provide two or more APIsets for each platform in the case of Tizen.

The application 2370 may include, for example, a home 2371, a dialer2372, SMS/MMS 2373, an instant message (IM) 2374, a browser 2375, acamera 2376, an alarm 2377, a contact 2378, a voice dial 2379, e-mail2380, a calendar 2381, a media player 2382, an album 2383, a watch 2384,and a health care (e.g., measure the quantity of motion or blood sugar)or environment information (e.g., atmospheric pressure, humidity ortemperature information) provision application. In accordance with oneembodiment, the application 2370 may include an information exchangeapplication capable of supporting information exchange between anelectronic device and an external electronic device. The informationexchange application may include a notification relay application fordelivering specific information to an external electronic device or adevice management application for managing an external electronicdevice, for example. For example, the notification relay application mayrelay notification information, generated from a different applicationof an electronic device, to an external electronic device or may receivenotification information from an external electronic device and provideit to a user. The device management application may control the function(e.g., the turn-on/turn-off of an external electronic device itself (orsome components) or brightness (or resolution) of the display) of anexternal electronic device communicating with the electronic device, forexample, and may install, delete or update an application operating inan external electronic device. In accordance with one embodiment, theapplication 2370 may include an application (e.g., the health managementapplication of a mobile medical device) selected based on the attributesof an external electronic device. In accordance with one embodiment, theapplication 2370 may include an application received from an externalelectronic device. At least some of the program module 2310 may beimplemented (e.g., executed) as software, firmware, hardware (e.g., theprocessor 2210), or a combination of at least two of them, and mayinclude a module, program, routine, instruction sets or a process forperforming one or more functions.

The term “module” used in this document includes a unit includinghardware, software or firmware, and may be interchangeably used with aterm, such as logic, a logical block, a component or a circuit. The“module” may be an integrated part or a minimum unit that performs oneor more functions or part thereof. The “module” may be implementedmechanically or electronically, and may include, for example, anapplication-specific integrated circuit (ASIC) chip, field-programmablegate arrays (FPGAs) or a programmable logic device, which performs someoperations and which has been known or is to be developed. At least someof a device (e.g., modules or functions thereof) or a method (e.g.,operations) according to various embodiments may be implemented asinstructions stored in computer-readable storage media (e.g., the memory2130) in the form of a program module.

When the instructions are executed by a processor (e.g., the processor2120), the processor may perform a function corresponding to theinstructions. The computer-readable storage media may include a harddisk, a floppy disk, magnetic media (e.g., magnetic tape), optical media(e.g., CD-ROM, a DVD and magneto-optical media (e.g., a flopticaldisk)), and embedded memory. The instructions may include code producedby a compiler or code capable of being executed by an interpreter. Themodule or program module according to various embodiments may include atleast one of the aforementioned elements, may omit some of the elements,or may further include another element. Operations executed by themodule or program module or another element according to variousembodiments may be executed in a sequential, parallel, repetitive orheuristic manner, or at least some of the operations may be executed indifferent order or may be omitted or another operation may be added tothe operations.

The invention claimed is:
 1. An electronic device, comprising: a cameramodule; at least one processor electrically connected to the cameramodule; and a memory electrically connected to the processor, whereinwhen the electronic device operates, the memory stores instructionsenabling the processor to: generate an image using the camera module, tocalculate angles formed by a plurality of straight lines and a virtualhorizontal line included in the image, and to select two straight linesbased on the calculated angles, select a center point of an areacomprising the selected two straight lines, and select a location of anobject detection area in which another transport apparatus is detectedbased on a location of the selected center point or to change a locationof a preset object detection area.
 2. The electronic device of claim 1,wherein when the electronic device operates, the memory storesinstructions enabling the processor to calculate two angles, belongingto the angles and having a preset value in a difference between absolutevalues, in order of greater absolute value and to select the virtualhorizontal line and the two straight lines forming the two angles. 3.The electronic device of claim 1, wherein when the electronic deviceoperates, the memory stores instructions enabling the processor toperform at least one piece of image processing on the image generatedusing the camera module and then detect the plurality of straight lines.4. The electronic device of claim 1, wherein the at least one piece ofimage processing comprises image processing for converting the imageinto a black and white image, image processing for extracting an edgefrom a black and white image, image processing for eliminating noiseincluded in an image, or image processing for detecting a boundary lineof an object included in an image.
 5. The electronic device of claim 1,wherein: the electronic device establishes a communication relation witha transport apparatus, and when the electronic device operates, thememory stores instructions enabling the processor to select the area,comprising the selected two straight lines, as a lane in which thetransport apparatus is driving.
 6. The electronic device of claim 1,wherein when the electronic device operates, the memory storesinstructions enabling the processor to set the object detection area ina quadrangle shape of a preset size.
 7. The electronic device of claim6, wherein when the electronic device operates, the memory storesinstructions enabling the processor to overlap the set object detectionarea with an image generated through the camera module and to detectanother transport apparatus in a portion included in the objectdetection area of the image.
 8. The electronic device of claim 1,wherein when the electronic device operates, the memory storesinstructions enabling the processor to calculate the angles, formed bythe plurality of straight lines and the virtual horizontal line, withrespect to a preset area of the image.
 9. The electronic device of claim3, wherein: the image generated using the camera module comprises aplurality of image frames, and when the electronic device operates, thememory stores instructions enabling the processor to perform the imageprocessing process on each of the plurality of image frames and then todetect the plurality of straight lines.
 10. A method of controlling anelectronic device, comprising: generating an image using a cameramodule; detecting a plurality of straight lines included in thegenerated image; and calculating angles formed by the plurality ofstraight lines and a virtual horizontal line and selecting two straightlines based on the calculated angles; selecting a center point of anarea comprising the selected two straight lines; and selecting alocation of an object detection area in which another transportapparatus is detected based on a location of the selected center pointor to change a location of a preset object detection area.
 11. Themethod of claim 10, wherein calculating angles formed by the pluralityof straight lines and a virtual horizontal line and selecting twostraight lines based on the calculated angles comprises: calculating twoangles, belonging to the angles and having a preset value in adifference between absolute values, in order of greater absolute value,and selecting the virtual horizontal line and the two straight linesforming the two angles.
 12. The method of claim 10, wherein detecting aplurality of straight lines included in the generated image comprisesperforming at least one piece of image processing on the generated imageand then detecting the plurality of straight lines.
 13. The method ofclaim 10, wherein: the electronic device establishes a communicationrelation with a transport apparatus on which the electronic device isput, and the method comprises selecting an area, comprising the selectedtwo straight lines, as a lane in which the transport apparatus on whichthe electronic device is put is driving.
 14. The method of claim 10,comprising calculating the angles, formed by the plurality of straightlines and the virtual horizontal line, with respect to a preset area ofthe image.